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ABSTRACT 



The Bureau of the Census conducts the Schools and Staffing 
Survey (SASS) for the National Center for Education Statistics. The SASS is a 
set of integrated self -administered surveys designed to obtain national 
information on all aspects of the school system. Public and private school 
administrators and teachers respond to questionnaires about their schools. 
When the early SASS questionnaires were developed, there was little 
information about the design of self -administered questionnaires, but more 
recent research, much of it focused on SASS questionnaires, has suggested 
that the look of a questionnaire could be as important to the quality and 
quantity of data collected as the wording and sequencing of the questions 
themselves. Research was conducted to study the impact of questionnaire 
format for the most recent SASS design and to determine how well respondents 
comprehended the questions. Twenty cognitive interviews were conducted with 
teachers, five of whom held alternative certification. The most problematic 
questions were identified, and results of transcribed discussions about these 
items are presented. (Contains 3 tables, 5 figures, and 16 references.) ( SLD) 
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Report of Cognitive Research on the 
Public and Private School Teacher Questionnaires 
for the Schools and Staffing Survey 1993-94 School Year 
by Cleo R. Jenkins, Center for Survey Methods Research 



Introduction 

The Bureau of the Census conducts the Schools and Staffing Survey (SASS) for the 
National Center for Education Statistics (NCES). SASS is composed of a set of 
integrated self-administered surveys designed to obtain national information on all 
aspects of the school system. Selected public school districts are mailed self- 
administered questionnaires, along with both selected schools from within these districts 
and their administrators. Also, private schools and their administrators are sent 
questionnaires, as well as selected teachers from the schools in sample. The survey was 
most recently conducted in 1993-94, and is currently scheduled to be conducted in 1999- 
2000. 

When the early SASS questionnaires (before the 1993-94 school year) were developed, 
there was little research in the survey field to effectively guide the design of self- 
administered questionnaires. In those days, survey research primarily focused on 
question wording and sequencing. As a result, early questionnaires were patterned after a 
questionnaire design that was originally intended for interviewers. However, over the 
course of time, a small body of research began to accumulate, much of it from cognitive 
research on the SASS questionnaires (Bates and DeMaio, 1990; Jenkins and DeMaio, 
1990; Jenkins et al., 1992a and 1992b; Jenkins, 1992; Jenkins and Dillman, 1993) and 
some of it from research with other questionnaires (e.g., Bates and DeMaio, 1989; Gower 
and Dibbs, 1989; DeMaio and Jenkins, 1991). This research began to suggest that the 
look of a questionnaire could be every bit as important to the quantity and quality of the 
data collected as the wording and sequencing of the questions themselves. 

The 1993-94 SASS questionnaires took into account some of this research. While both 
the early SASS questionnaires and the 1993-94 questionnaires were all booklets, the 
booklets differed in the way they made use of color. For instance, the cover pages of the 
early SASS booklets were colored, but the inside pages were entirely off white with black 
print. The answer spaces blended in with the rest of the page. In the 93-94 
questionnaires, the background of each page was colored (e.g., the Private School 
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Teacher Questionnaire used blue and the Public School Teacher Questionnaire used 

green), but the answer spaces were left white. This was done to offset the answer spaces. 4 

Another major change was in the layout of the questions and answers and the layout and 

coloring of the source codes. The previous questionnaires used a two-column format, 

with the questions in the left-hand column and the answers in the right-hand column. ^ 

Also, the source codes were prominently displayed in front of the answer columns. They 

were printed in black (the same as the rest of the print) and outlined in a black box. The 

93-94 questionnaires made use of a one-column vertical format, with the questions on top 

and the answer spaces vertically aligned beneath. The source codes were placed to the far 

left of the page, out of the respondent’s view, and they were put in the same color as the <1 

background to de-emphasize them. 

A third change was in the way the skip instructions were designed. The original SASS 

questionnaires used a conventional skip instruction, an example of which is given in 

Figure 1. The 1993-94 attempted some newly designed skip instructions, termed the ^ 

intermediate and salient skip instruction. Shown in Figures 2 and 3, these will be 

discussed in greater depth below. 

As noted above, the NCES has a long history of sponsoring cognitive research on the ^ 

SASS questionnaires in a continuing effort to pretest and improve the questionnaires. In 
this latest round, NCES asked the Center for Survey Methods Research to conduct 
cognitive research with the 1993-94 Public and Private School Teacher Questionnaires. 

The goal was two- fold: to determine how effective the questionnaire’s new format was 

and to determine how well respondents comprehended the questions. 4 



Methodology 

Twenty cognitive interviews were conducted with teachers, distributed by school type and ^ 

teaching experience (Table 1). Fifteen of the interviews were conducted either at libraries 
in the Washington-DC Metropolitan Area or at the Census Bureau’s cognitive laboratory. 

The answers of teachers with alternative certificates to question 22 b (type of certificate) 
were deemed especially problematic by NCES. As a result, five interviews were 4 

conducted at libraries in New Jersey with teachers who were identified by the Department 
of Education as having alternative certificates. 

A combination of cognitive techniques were used-including the concurrent think-aloud 
technique, the use of paraphrasing, and unstructured retrospective interviewing. • 
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t Respondents were given the Teacher Questionnaire to complete. They were asked to read 

aloud as they read through the form and to think aloud as they answered the questions. 
The interviews took from Vi hour to 1 l A hours. With the respondent's permission, the 
interviews were tape-recorded and either a summary or a transcription of each was 
written. 

NCES identified questions 6, 9, 10, 11, 13, 22, 30 through 33, and 52 on the questionnaire 
as being the most problematic. Because pages 20 through 28 and 30 through 34 didn’t 
contain any of these questions, these pages were stapled together and covered with a 
m piece of paper that read “This page has been blanked out for research purposes.” 



Summary and Analysis 



9 Formatting Issues 

Color and Vertical layout . Using color to distinguish the background of the questionnaire 
from the answer spaces and laying out the questionnaire in a one-column format worked 
well. However, the lines separating the questions from one another clutter the page. We 
* want respondents to move freely from one question to another, whereas the lines act like 

stop signs. 



^ Recommendation: The color scheme should be maintained. However, one change 

that might decrease the length of the questionnaire, and thereby, 
increase response rates, without harming the navigational qualities 
of the questionnaire would be to layout the questionnaire with two 
vertical columns per page. A good example is provided by the 
• National Survey of College Graduates (NSCG). Although 

respondents did not have problems reading across the width of the 
Teacher Questionnaire, we might be able to ease their task even 
more if we limit their eye movements to half the page, the way the 
„ NSCG does. Note that the NSCG begins with A1 in the upper left 

hand comer. In the Teacher Questionnaire, the first and second 
instructions should be combined and correctly labeled the first item 
on the Teacher Questionnaire. The Teacher Questionnaire is 
sending mixed signals to respondents. On the one hand, it implies 
9 that the beginning of the questionnaire begins at the top of the 

page. However, question 1 is midway down the page. All of the 
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information should be working in unison so that respondents are 4 

not confused about where to begin. I would suggest putting a 

heading in the upper left hand comer that says “Start Here.” Then 

beneath that I would put the number 1 . This way there is no 

confusion about where to begin. I would also recommend that the 

lines separating questions be removed, since they are interfering ® 

with progression through the form. 



Skip Instruction Problems . Skip problems should be interpreted with care for two ^ 

reasons. First, this research was conducted with a small, purposive sample of 

respondents, so care must be exercised in generalizing beyond this research. Secondly, 

many features varied uncontrollably among the questions with skip instructions (as 

shown in Table 2), which makes it difficult to identify with any precision or reliability 

the reason for the response differences. However, as noted below, respondents • 

appeared to overlook skip instructions placed to the right of an answer category (or 

categories). Also, they had difficulty with some implicit skip instructions. Finally, a 

few respondents totally misunderstood the skip instructions. 

Problems with Skip Instructions Placed to Right of Answer Categories ® 

Shown in Figures 2 and 3 respectively, respondents overlooked the skip 

instructions in questions 1 (main assignment) and 6 (main assignment year 

before) more than in 9 (public teacher) and 15d (second major). (These are the ^ 

questions with skip instructions that were asked of everyone. A few respondents 

did answer other questions with skip instructions, but the sample size for these 

questions was even smaller, making it even more difficult to draw conclusions.) 

Table 2 shows that about a third of the respondents (6 to 7) misexecuted the skip 
instruction in questions 1 and 6 (the main assignment questions) compared with a • 

tenth or less (0 to 2) who misexecuted the instructions in questions 9 and 15d 
(the public school teacher and second major questions). 

The wording, location, and context differed among all of these questions. 

However, as can be seen in Figures 2 and 3, one of the most obvious differences ® 

between the main assignment questions versus the public teacher and second 
major questions was that the main assignment questions were more complex. 

The more complex questions had a long list of response categories, whereas the 
simpler questions had only two: “yes” and “no.” The more complex questions ^ 

tended to use a bracketed arrow that came off a number of the response 
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categories and narrowed to the “skip” instruction or went to the next question, 
whereas the simpler questions tended to have a single arrow that came off each 
of the response options. One of the arrows went to the next question, the other 
to a “skip” instruction. In both cases the background of the “skip” instruction 
was darker than the background for the rest of the page; however, the location of 
the arrow and instruction differed between the two groups. In the complex 
questions, the skip was off to the right of the answer category (or categories). In 
the simple questions, it was located beneath the “no” answer category. 

Question 11a (breaks in service) was an interesting hybrid because it exhibited 
both kinds of features. This question is shown in Figure 4. It was like the 
simple questions in that it had only two response categories. Also, it was like 
the simple questions in that only a single arrow came off of each of the answer 
categories. However, it was like the complex questions in that the arrow and 
skip instruction was off to the right of the answer category and although there 
were only two response options, they weren’t just “yes” and “no.” They were 
“none” and a write-in answer space “[ ]” for number of breaks in service. 
Although the breaks-in-service question had features of both the complex and 
simple questions, practically the same number of respondents mishandled its skip 
instruction as mishandled the skip instruction in the complex questions. 

As just mentioned, there were two major differences between the breaks-in- 
service question and the other complex questions; that is, the response options 
differed (yes/no versus none/write-in) as well as the location of the skip 
instructions. Although it is impossible to disentangle these differences, we can 
look at other research to provide additional insight. Dillman et al. (1996) 
conducted cognitive interviews with a question on the decennial short form that 
contained a skip instruction (household elsewhere). This question is shown in 
Figure 5. Comparing the household-elsewhere question in Figure 5 with the 
breaks question in Figure 4 may provide some information about the differences 
in the wording of the response options. Comparing the household-elsewhere 
question in Figure 5 with the second-major question in Figure 3 may provide 
some information about the differences in the location of the skip instruction. 
However, it should be noted that these are not the only differences in these 
questions. The wording, location, and context varies among the questions, but 
this is true of all of the comparisons, not just these two. In addition, there are a 
couple of differences that are specific to these comparisons; that is, the response 
categories were horizontally aligned on the decennial form, whereas they were 
vertically aligned on the Teacher Questionnaire. In addition the graphical style 



of the questionnaires differed (e.g., the background color, font size and type, 

etc.). 

Ignoring these differences, however, and comparing the breaks question with the 
household elsewhere question from the decennial short form, where the 
difference of interest is the response categories, it can be seen in Table 2 that 
nearly the same percentage of respondents misexecuted the skip instruction in the 
household elsewhere question on the decennial short form (25%) as in the breaks 
question (25%). Comparing the breaks question with the second major question 
shows that a smaller percentage of respondents misexecuted the skip instruction 
in the second major question (10%). Although it is difficult to draw conclusions 
from this research for the reasons mentioned above (small purposive sample, 
uncontrolled variables, etc.), these findings do seem to suggest that placing the 
answer category off to the right of the answer category is the real problem here, 
not the difference in answer categories (yes/no versus none/breaks). 

It has been suggested that overlooking skip instructions in a concurrent cognitive 
interview may be attributable to the concurrent interviewing method rather than 
the skip instruction. The argument is that the interactive nature of the 
concurrent interview distracts respondents. That is, the act of talking about the 
question and/or answer interferes with respondents’ perceiving the skip 
instruction. Although this is a plausible competing hypothesis, a growing body 
of prel iminar y data suggest that the concurrent interview is not solely responsible 
for respondents overlooking skip instructions. If the interviewing method is 
solely responsible for this phenomenon and the placement of the skip instruction 
plays no role in this, then we would expect practically the same percentage of 
respondents to misexecute all of the skip instructions in a concurrent interview, 
irregardless of where the skip instruction is placed in an item. This does not 
appear to be the case. As pointed out above, it appears that respondents have a 
tendency to overlook skip instructions placed to the right of an answer category 
more than those placed beneath the “no” answer category. 

Even more convincing evidence that the cognitive interviewing method is not 
responsible for respondents overlooking skip instructions comes from research 
with the decennial short form. In the Dillman et al. (1996) study mentioned 
above, half of the interviews were conducted using the concurrent method and 
half using the retrospective method. In a preliminary analysis (unpublished as of 
yet), the number of persons who overlooked the skip instruction in the two 
methods was similar. 



Further evidence refuting the notion that the concurrent interviewing method is 
responsible for respondents overlooking skip instruction comes from a study by 
Turner et al. (1992). They concluded that respondents only see information off 
to the right of an answer category if it is in some way made salient. Since they 
conducted a mailout/mailback experiment, the cognitive interviewing method 
could not in any way be responsible for their conclusion. 

Jenkins and Ciochetto (1992) concluded that respondents overlook skip 
instructions that are placed to the right of an answer category because of the 
convoluted reading structures these instructions present. The researchers assert 
that respondents read the answer category, move to the left to mark the answer 
box, and that their natural inclinations at this point are to go to the next question 
rather than back to the right to read the skip instruction. Given this information, 
Jenkins and Dillman (1993) discussed what they foresaw as the advantages and 
disadvantages of three skip instructions: the salient skip instruction, the 
intermediate skip instruction, and the natural reading sequence skip instruction. 

• In the salient skip instruction , the instructional arrows came off the 
left-hand side of the answer boxes. One arrow pointed to the next 
question, while the other pointed to the skip instruction. In 
addition, the skip instruction was placed beneath the answer 
category (rather than to the right of it) in an effort to make the skip 
instruction more visible to respondents. This was the skip 
instruction used in questions 9 and 15d of the Teacher 
Questionnaire. 

• The intermediate skip instruction differs from the salient skip in 
that the one of the arrows and its associated skip instruction was 
placed to the right of the answer category. This skip instruction 
was used in question 1 of the Teacher Questionnaire. 

• Finally, Zukerberg and Hess (1996) conducted cognitive interviews 
with the natural reading sequence skip instruction . In this format, 
the information was presented to respondents in the sequence they 
seemed to need it: first, the answer category, then the answer box, 
and finally, the arrow and instruction. The researchers reported 
that a little over half (11 out of 20) of the respondents misexecuted 
at least one of the natural reading sequence skip instructions. 
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Jenkins and Diliman (1993) discussed what they saw as the advantages and 
disadvantages of each of these skip instructions, but theirs was a hypothetical 
discussion unsubstantiated by much experimental evidence. Although we still 
lack additional experimental evidence on which to base conclusions, it would 
seem that taking together all of the above preliminary evidence, the salient skip 
instruction is the best skip instruction to use when there are only two response 
categories. The question remains whether or not the salient skip instruction 
would work better, equally as well, or worse than the intermediate skip 
instruction when there are more than two response categories. Since no such 
skip instruction has been pretested, it is impossible to say at this point. 
However, based on the above discussion, it certainly appears worth exploring 
further. 

Recommendation: At this point in time, the salient skip instruction looks 

the most promising in the case of a simple two- 
response category question. Also, it seems worth 
exploring its use in the case of multiple response 
options. Beyond that, however, experimental 
research is needed to further elucidate this problem. 

The earlier versions of the Teacher Questionnaire did not 
attempt to highlight the skip instructions. They just took it 
for granted that respondents would perceive the skip 
instructions. We have since learned that this is not 
necessarily the case. As a result, the 1993-94 questionnaire 
tried to highlight them by putting them in a deeper 
background than the rest of the questionnaire. However, 
since then, we have come to learn that black print is more 
visible against a lighter background than a darker 
background. Given that respondents are still having 
problems perceiving the skip instructions, we should better 
highlight them by putting them in a white background rather 
than the dark background they are now in. 

Problems with Implicit Skip Instructions 

Although all of the respondents executed the explicit skip instruction in Q9a 
correctly, a number of respondents had difficulty with the implicit instructions 
that followed in Q9b, and Q9c, and in Q 10a and QlOb. After marking “yes” in 
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Q9a, and answering Q9b, they weren’t sure if they were supposed to go to Q9c. 

* Coupled with the fact that they hadn’t taught part-time, a number of respondents 
decided to leave Q9c blank rather than mark the “none” box. In addition, one 
respondent thought that she was only supposed to answer Q 1 0 if she responded 
“no” to Q9a. So, she left QlOa and QlOb blank and went to the next page. 

* Recommendation: Q9a should be broken into two separate questions. Q9a 

would become: Have you ever worked as a FULL-TIME 
elementary or secondary teacher in a PUBLIC school? Q9b 
would still ask for the number of years. Q 1 Oa would become: 

9 Have you ever worked as a PART-TIME elementary or 

secondary teacher in a PUBLIC school? QlOb would ask for 
the number of years. The skip instruction would say to go to 
Qlla. 



Serious Skip Instruction Problems 

Three respondents had very serious problems with the skip instructions. Two of 
% these respondents thought that they were supposed to skip to the specified 

question in the skip instruction, but then return from whence they came. As a 
result, they found themselves in recursive loops and answered or attempted to 
answer many, if not all, of the questions, even when they weren’t supposed to. 
For example, Table 3 shows that Respondents 13 and 19 answered Qla, then 

• properly skipped to Q5. However, rather than continuing on to Q6 at this point, 
they went backwards. One of the respondents went to Qlb then to Q2, the other 
went directly to Q2. However, Q2 directed them to go to Q5. So, they went to 
Q5, recognized they had already completed it, and once again looped backwards. 

m One respondent went to Q3, the other to Q4. Eventually, they worked their way 

through all of the questions up to 5, at which time, they finally felt comfortable 
going on to Q6. 

The troubling aspect of this behavior is that it didn’t occur because respondents 

• were overlooking the skip instructions, but because they had read them and 

misunderstood them. This seems to be a special case of the above category, in that 
respondents are getting to Q5 and looping backwards because Q5's instruction to 
go forward to Q6 is implied rather than explicit. 
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Recommendation: These questionnaires would make excellent candidates ' 

for computer self-administered questionnaires because 

the computer will be able to control respondents’ paths 

more effectively than the respondents can. If the 

questionnaires remain paper, however, it would be 

advantageous to reduce the number of questions with 9 

skip instructions in the Teacher Questionnaires. Also, 

every question needs to have an explicit instruction 

telling respondents to move ahead. So, for instance, an 

arrow would come off the left hand side of the answer 

box in Q5 and go to Q 6 . * 



Item by Item Review of Problematic Questions 

■ • 

6a. What was your MAIN activity the year before you began teaching at the 
elementary or secondary level? 

The sixth answer category ‘Working in a position in the field of education’ gave 

respondents the most trouble. One respondent was a tutor. He questioned whether 

this should be reported in ( 6 ) Working in a position in the field of education, but 

not as a teacher or in (7) Working in an occupation outside the field of education. % 

He couldn’t decide if tutor was equivalent to teacher. He noticed, however, that 

whatever he decided would effect what questions he answered next, so he looked 

to see if the future questions would provide him with some guidance. He decided, 

although he wasn’t satisfied with this, to mark ( 6 ) Working in a position in the 

field of education, but not as a teacher in Q 6 a and ( 6 ) Instructional aide in Q7. In 

addition, another respondent had difficulty determining how she should report her 

position as a teacher’s aide. She wasn’t certain if it should be reported in ( 6 ) 

Working in a position in the field of education, but not as a teacher. She decided it 
should be. 4 

Recommendation: Make it clear how a tutor should be reported. Also, 

combine questions 6 and 7. This will eliminate the 
need for respondents to interpret the sixth category. 

It’ll also eliminate the need for an additional question 4 
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and skip instructions, which only allows more room for 
error. 



9a. 1' (Private School Questionnaire) Have you ever worked as an elementary or 



9a. (Public School Questionnaire) Have you ever worked as an elementary or 



secondary teacher in a PRIVATE SCHOOL? 



A number of respondents wondered if this question was referring to part-time, 
full-time, or both. For instance, one private school teacher had substitute taught 
at a public school, which she considered part-time teaching. She read aloud 
Q9a. Since she couldn’t tell if it referred only to full-time teaching, she went on 
to part b to see if it would provide her with guidance. Because part b referred 
only to full-time teaching, the respondent decided that part a was only referring 
to full-time teaching too. Therefore, the respondent erroneously marked “no” in 
response to Q9a, left b blank, and went on to answer part c correctly. This 
resulted, however, in inconsistent answers between the parts of this item. 



One respondent expressed difficulty with the word “work. ” The respondent had 
observed as part of a college course and wondered if that constituted “worked.” 
He decided it didn’t because he hadn’t been paid. 

Recommendation: As noted earlier, break Q9a into two separate questions. 
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^ PART-IIME elementary' or' secondary teacher in PUBLIC 

delude the: current. school year if you are a full-time teacher this year. 

jjjjpj|^^ 

10b. (Private School Teacher Questionnaire) How many years have you worked as 

Include the current school year if you are a-full-time teacher this year. 

Record whole years; not fractions qrjnonths. Ifless than 4 months,, mark “None. ” 



A couple of respondents overlooked the “none” box altogether. One of these 
wrote a “0” in the write-in answer space, which didn’t result in a loss of data, 
but another left the item blank, which did result in a loss. Another respondent, 
however, did the opposite: she wrote the number of years she had been teaching 
in the “none” box rather than the “write-in” box. 

One respondent worked in two schools and wondered if he was supposed to 
answer question 10 in terms of both schools. He decided to report the eight 
years he had worked full-time at “this school” in part a, and the twelve years he 
had worked part-time as a tutor in a parochial school in part b. Notice that for 
this question the respondent equated “tutoring” with “teaching.” 

Recommendation: Respondents who wrote in “0” rather than marking the 

“None” box were looking for a “0” so the word “None” did 
not register. Therefore, substitute a “0” for the word “none.” 



Ifliill school? Hill! i 



This question gave respondents the most difficulty. Visually, this item clearly 
demonstrates the value of the Gestalt principles. This item asks respondents what 
their main activity was the year before they began teaching at their current school. 



It also provides them with an instruction about what to do if they left the school 
and then returned. If their main activity is any of the first four answers, the 
bracketed skip instruction off to the right instructs respondents to go to item 15a, 
and if it’s any of the next 10 answers, the bracket along with the arrow off to the 
left directs them to go to item 14. However, the large white answer box between 
the fourth and fifth answer categories, along with the way the brackets divide up 
the first through fourth answer categories from the fifth through fourteenth answer 
categories made this item look like two separate questions to respondents, as will 
be demonstrated shortly in the transcript below. Notice that the respondent 
answered this question, and in fact he answered it quickly, but he continued to 
struggle with it after he answered it. He explained that we should have put a 
dotted line after the big white answer box, and that we should have put 13a off to 
the left. That’s because at this point he thinks that this item is two different 
questions, but then he happened to notice that the keycodes for the answer 
categories are part of a continuing series and this clued him into the fact that this 
may be one question after all. 

Alright then it goes on to say, what was your main 
activity the year before you began teaching at this 

school, meaning, my current school . Okay I 

was, okay if you left it before, if you left and then 
returned, it says report the main activity the year 
before you most recently returned to this school . 

Okay, I haven't left it. So I don't worry about that. 
So I just cross it out. I see there's only one box. 
Okay, I was teaching in another elementary or secondary 
school in this school system. Right. So, then, then I 
mark it. Then I say different school system. No. 

Then I just read through it to make sure. Nope. 

Teaching in a private school. So it said mark only one 
box, and that one fit, so that was good. 

Okay, now I'd have to go, since I've answered this 
question, that's nicely delineated there. It might be 
nice to have a dotted line maybe. Or put 13a here, or 
something like that. It's too complicated. It's 
getting. . .for my old mind, it's getting too 
complicated. Just have something there. 

Alright, go to, because it looks like I should be 
going. 'Cause it says mark only one box, then I check. 



Then I have to go back and check and see if that 4, if 
there's a 1, 2, 3, 4. Because I just assumed, since I 
wrote this in, this is one, two, three, four. Now I 
see it's all part of the same question. 

So what impeded my understanding was that big box 
there, the big box. Visually and spatially it 
uhh. . .visually and spatially, maybe it should be over 
here, and maybe go to item 15 should be over here. But 
it, it... Anyway it confused me. Alright, so I don't 
have to worry about... wait a minute I do have to worry 
about question 14, no, I don't. Go to item 15a, page 
11. At least I read question 14, and realized then I 
had a bracket out here, so that helped. So I just go 
on. [Interview 9] 

Technically question 13 is not the same as question 6 because 13 asks respondents 
what their main activity was before they began teaching in this school, whereas 6 
asks what their main activity was before they began teaching ever. However, for 
new teachers, questions 13 and 6 were one and the same (except that the answer 
categories for the two differed slightly). As a result, many respondents answered 
13 differently than they had 6, even though they shouldn’t have. 

To understand why this occurred, we need to consider the assumptions that govern 
the conduct of conversation in everyday life. One such assumption is that speakers 
are supposed to provide new information to recipients, rather than reiterate 
information that they’ve already provided (Schwarz, 1993). It is only natural, 
therefore, that respondents thought they were supposed to provide “additional” 
information in question 13. An example of this follows. 

The respondent read Q6, "What was your main activity 
the year before you began teaching at this." The 
respondent reread "My main activity." She continued to 
read the first two answer choices, and then went back 
to read the instruction, "Mark only one." She read the 
list of answer choices, stating that she was not a 
housekeeper, but did however, work as a substitute 
teacher. She continued to read the answer categories, 
pausing on the answer "working in a position in the 
field of education." She then stated, "not as a 
teacher, no." When the respondent read the answer, 



"working in an occupation outside the field of 
education, " she said that she could have two possible 
answer choices. So I asked what her two answer choices 
were, and she said that she was a substitute teacher, 
but her primary job was in advertising. She wondered 
how she would answer this question. She decided that 
the question wanted to know about her history. She 
then decided to answer "working in an occupation 
outside the field of education". I asked her why she 
chose that answer, and she responded that she thought 
the question wanted to know where she came from as a 
teacher. All the other answer categories dealt with 
being involved in school and she was not entirely 
involved with the school. 

The respondent read Q13, "What was your main activity 
the year before you began", and thought this question 
was already asked. [She was referring to Q6.] She 
reread the phrase, "main activity" and stated that this 
was where she was going to throw the questionnaire a 
curve, and answered substitute teaching. She read the 
first answer choice aloud, but glanced through the 
other answer choices without reading them aloud. She 
decided that she would answer "teaching in another 
elementary or secondary school in this school system" 
because she had a double background and wanted to let 
the questionnaire know that she did. [I believe she 
was referring to substitute teaching and working in 
advertising.] I asked her to explain more. She 
explained that substituting became more important 
because she was making money. If she was offered a 
substitute job, she always took it, and did not do the 
advertising. [--Interview 20] 

As mentioned earlier, the answer categories in this question differed from 
the answer categories in Q6. The first four were unique to this question and 
the remaining answer categories, which were the same as the ones in Q6, 
were arranged differently than in Q6. This threw new teachers off. They 
were expecting the answer categories to be the same in Q13 as in Q6, since 
they perceived these as the same questions. 
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Recommendation: Do not ask this question of respondents for whom this 

is the same question as question 6. Also, move the 
white answer box to the right of the answer category in 
4 because it is interfering with respondents’ perceiving 
this as one question. 



v ^ /-grades ^;K 

through 12? : , 

Donot-j^^de student teat&yte 



One respondent demonstrated just how interesting definitions can be. He said that 

although he substitute taught over a long period of time, technically it was 

considered short term teaching. With probing, he explained that short-term ® 

substitute teaching was considered less than 20 consecutive days in the same 

classroom. He explained that his long-term stint was interrupted by the fact that 

the regular classroom teacher came back for a day. Therefore, what might be 

considered a long-term substitute teaching job by some standards might become a g 

short-term one according to the school’s definition. He marked “no,” which I 

assume was the correct answer here. 

Although he answered Q14 correctly, he hadn’t answered Q13 correctly. There he 

had marked 3 categories rather than 1 . The three categories were: (5) Working as • 

a substitute teacher, (8) Working in a position in the field of education, but not as a 

teacher, and (11) Student at a college or university. However, we learned in Q6 

that he should have marked (11) Student at a college or university as his main 

activity the year before he began teaching, which was also the year before he 

began teaching in this school. Hopefully, the fact that he marked three categories 

in Q13 will be corrected when he is no longer asked this question, as explained 

above. 

However, his response brings to light two additional problems. What if his main g 

assignment had been short-term substitute teaching. Was he supposed to mark the 
(5) Working as a substitute teacher. It is not clear if (5) is referring to both short- 
and long-term substitute teaching. He interpreted it as referring to short-term 
substitute teaching because he hadn’t been told anything different up till this point. 

Also, let’s assume his answer to Q13 was (5) Working as a substitute teacher. • 

Then it doesn’t make sense to ask Q14. This respondent answered Q14 in light of 
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the substitute teaching job he reported in Q13, not “in addition to the above 
assignment” as Q14 directs. 

Another respondent had a similar problem. She too ignored the reference to “in 
addition to the above activity” and answered Q14 in light of the activity . Also, 
she thought the question was asking her if she taught in a K through 12 school, and 
since she had taught in a K through 8 school, she responded “no” to this question. 
Her answer ended up being correct, but for the wrong reasons. Her answer was 
correct because she hadn’t taught “in addition to the above activity.” 

Another respondent temporarily forgot about her other part-time teaching 
assignment for two reasons: one, because it had occurred 8 years ago and two, 
because it wasn’t her main activity at the time. 



Recommendation: Some of the problems with this question should be taken care 

of by not asking new teachers question 13. In addition, 
reword category (5) so that respondents know if its referring 
to short-term substitutes, long-term, or both. Finally, exclude 
the fifth category from the skip bracket on the left of Q13 
going to Q14. 



15b. What was your major field of study? 

Pill 



There was a little difficulty with the list of field codes. For instance, one 
respondent thought that she was supposed to choose “General Education” until she 
realized that there were more specific fields and codes underneath this heading 
from which to choose. A French teacher’s eye went immediately to “24— Foreign 
languages education.” No sooner had he written in “24” than he realized that 
wasn’t correct because he hadn’t gotten a degree in foreign language education, he 
had gotten it in “French.” So he correctly changed his answer from “24” to “51.” 
And another thought that the list was “...two different lists instead of one long list.” 
She thought one side was the code and the other side looked like the main field of 
study. Eventually she figured it out. 



o cr 
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Recommendation: Visually highlight the headings and differentiate the columns, 

so that it becomes more evident that the one column is dealing 
with education fields and the other with general fields. 

Illlltll 15e; :?^at was your second major field of stndy? ^ ^ ^ 4 

llllf the two-digit field codejrongt^e lisi : '.on page Wj^^iefield name. . 

Quite a few respondents thought this question was asking if they had a minor at 
first. Some respondents looked ahead and saw that the next question (Q15f) 
actually asked about their minor. Then they realized that this question was asking 
them if they had a second major. Others actually answered this question in terms 
of their minor and had to erase it when they got to Q1 5f. 

Recommendation: Capitalize the words ‘SECOND MAJOR’ in Q15d and 

“MINOR” in Q15f. 



16a. ' ;• Wbatis the same of the college or university where you earned your 
! bachelor’s degree? ,= • >-< 



The answer format of this question violated respondents’ expectations. Starting at 
the top of page 11 in Q 15b, the questionnaire asks respondents to record the two- 
digit field code from the list on page 10 in white segmented boxes that are placed 
to the left of a long write-in answer space. What happens in question 1 6 is that 
after having asked respondents to write in field codes in questions 1 5b, e, and g, 
the questionnaire turns around and provides the same white write-in answer boxes 
for another purpose. Instead of being provided for respondents to write in, they are 
being provided for office personnel to write in. 

Top-down processing enables us to quickly perceive our world by placing what are 
well-founded expectations upon it from past experiences. However, a break down 
is likely to occur when the world deviates from our expectations, as it does here. 
According to the caption in 1 6a, the segmented boxes on the left are for office use 
only. 

Recommendation: One suggestion is to remove these boxes altogether. Couldn’t 

coders be instructed to code these responses irregardless of 
whether there are boxes? Otherwise, move these boxes to the 
right hand side of the write-in answer space rather than the 
left. Also shade these boxes to give respondents the 
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impression that they are not supposed to pay attention to them 
(because they have learned to respond in white answer boxes, 
not shaded ones). 



at this school, that is, the field in 

* which yon 'teach 'the most classes? :i£!) 

| Record the two-digit codkfrbm the list above and the field name. Ifyoiir teaching 

^e : t^d^^^^^coj^^dher^^di^^our main. 

ft markfiXl b§boot l, and_ report the otkerfield in item 2/cp'fll£ 7 • 

Elementary teachers were confused by this question, especially when they neither 
read the instruction nor looked at the list of field codes. Although they primarily 
think of themselves as teaching all subjects equally, the phrase “teach the most 

* classes” made them question whether they were supposed to single out reading and 
math here. “The most classes” seemed to be interpreted as meaning “ the most 
important subjects.” After considering whether or not to report reading or math as 
her main teaching assignment, one respondent decided instead to write in 

ft “Classroom Teacher (elem.)” in the white answer space. However, because she 

neither read the instruction nor the list, she didn’t transpose a code from the list 
above. A different teacher figured out the correct meaning of this question once he 
saw the instruction which said, “if your teaching schedule is equally divided” and 
the code for “general elementary” from the list above. He then correctly wrote 

* “03— General elementary” in the white answer space. A third respondent said she 
would have been better off if we had told her on the previous page to turn the page 
and start at the top. Then, she said, she would have understood sooner that these 
questions applied to the field codes above. 

* However, for some reason there were also quite a few different kinds of errors 

made with this question. For instance, one respondent was confused about 
whether this question referred to her current posting. Another had difficulty 
understanding what the “If your teaching schedule is equally divided” instruction 

ft meant at first. Another respondent thought he had already been asked this question 

before. He must have been thinking of the question that asked him what his major 
field of study was, because his answer to both of these questions was the same: 51 
French. It is also interesting to note that one of the elementary teachers mentioned 
above found “mathematics” in the teaching assignment field code list, but not 

® “reading.” However, a mathematics teacher couldn’t find the field code for math. 
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He thought that math would come under “Science,” but it comes under “Special 
Areas.” 

Recommendation: Combine math with science and change the heading to “Math 

and Science.” Do not rely solely on the instruction provided 
below the question for respondents to refer back to the list 
because some respondents overlook it. Reword the question 
so respondents know to refer to the list provided: “From the 
list above, record the 2-digit field code to indicate your MAIN 
teaching assignment at this school, that is, the field in which 
you teach the most classes.” Right now the list looks visually 
disconnected from question 21a, especially because of the 
line between the two. Visually connect the two by removing 
the line. 



22b. What type of certificate do you hold in this field?: 



The problem with this question was that respondents who had alternative 
certificates and who were supposed to mark the fourth answer category “The 
certificate offered in your state to persons who HAVE COMPLETED what the 
state calls an ‘alternative certification program’. ” didn’t necessarily mark this 
category. Three of the five respondents marked the “regular or standard state 
certification” instead. One of these didn’t see the alternative certification category. 
This respondent engaged in a behavior known as satisficing (Krosnick, 1991). She 
came to a category that described her certificate well enough, chose that category, 
and moved on. 

The rest, however, purposely chose the “regular” category over the “alternative” 
category. Although they had received their certification through an alternate route, 
they made it clear that once they received the certificate, it was the same as any 
other teachers’. There was a reluctance on their parts to mark the fourth category 
for fear that made their certificate look different, or perhaps even inferior to, the 
regular certification. When respondents talked about this program, they didn’t call 
it “alternative certification program,” they called it “alternate route program.” 

Recommendation: One option is to reword the fourth category to “Regular or 

standard state certificate received through the ‘alternate route 



program’.” However, this is probably not the best solution, 
since it will probably not correct mistakes made by 
respondents who engage in satisficing. Another solution is to 
create a skip instruction here, such that respondents who mark 
the third category are then asked a follow-up question about 
the way in which the certificate was received. A third 
solution, and the one I recommend, is that the questionnaire 
not collect data about the alternate route program in this 
question. The program pertains to the teacher’s training, not 
certificate. A new question should be created to obtain this 
information. 



;:23ia. Do you have a teaching certificate in this state in your OTHER teaching: 

. •\L’:;assignmen|pletdi^ihis-school?' : ^||; : ;l|^^^^ : > 



When asked in Q21b if they taught classes in other fields at this school, most 
respondents marked “no,” yet several questions later they were asked if they have 
a teaching certificate in their other teaching assignment field. Not surprisingly, 
this question confused respondents because they seldom ever had an OTHER 
teaching assignment field in the school, but the question is worded as though they 
do. It doesn’t say “Do you have an OTHER teaching assignment field? Yes/No.” 
It says “Do you have a teaching certificate in your OTHER teaching assignment 
field?” Before reading the answer categories, and sometimes even after reading 
the answer categories, respondents thought they had overlooked or misunderstood 
something. Most respondents eventually figured it out, but we should not burden 
them this way. 

Recommendation: Reword and rearrange questions 21 through 23 so they flow 

more logically. 

2 1 . From the list above, record the 2-digit field code to 

indicate your MAIN teaching assignment at this 
school, that is, the field in which you teach the most 
classes. 



22. Do you have a teaching certificate in this state in your 

MAIN teaching assignment field? 

b. What type of certificate do you hold in this field? 

23a. Do teach classes in other fields at this school? 

b. From the list above, record the 2-digit code to 
indicate the field in which you teach the 
SECOND most classes? 

c. Do you have a teaching certificate in this state 
in your SECOND teaching assignment field? 

d. What type of certificate do you hold in this 
field? 



l 303 i lli ce the end of||ast 's& of these activities related to 

teaching have you participated? 

Iflf Mark (X) all that apply- . V'V: 



A couple of respondents weren’t certain if the word “participated” in this question 
was supposed to include teaching. For instance, one teacher taught a workshop 
and wondered if she should mark the second category, “SCHOOL sponsored 
workshops or in-service programs.” Another taught an adult education course, and 
wondered if she should mark “University extension or adult education courses.” 
Both decided they should. 

Recommendation: Reword the question as “Since the end of last school year, in 

which of these activities have you participated or taught. 
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9 Q SCHOOL DlpTRICT sponsored workshops.. on : m-sery 2 ce' : prp^ams {Public p|; : ; 

•' ^:J^doiTeach^ : .QuestiQiinai^ «'*. ' • ' :7 ^ 74*. > .. ; ' • P : $?■ -T'r ;; \ . 

181: jo SCHOCfe sponsored workshops or in-service programs 

^ A couple of public school teachers had difficulty because they couldn’t decide how 

to report state-affiliated activities. For instance, one respondent talked about 
taking courses which were for, as she put it, MSD credit. MSD stood for 
Maryland State Department. She decided that the MSD courses probably fell 
t under the school district sponsored workshops because the school district- 

sponsored them. Another respondent had taken a whole language workshop that 
she said was sponsored by the state. She looked for an answer category, but didn’t 
find one, so she didn’t mark any. Rather than seeing these answer categories as 
mutually exclusive, this respondent viewed the school sponsored category as a 

* subset of the school district sponsored category. 

One public school teacher had difficulty distinguishing between the first two 
categories on the Public School Teacher Questionnaire: the “SCHOOL 
9 DISTRICT sponsored workshop” versus the “SCHOOL sponsored workshop.” 

She marked both, saying “They are pretty much the same thing, so I would say 
‘yes’ because the school is controlled by the school district.” 

Recommendation: Including a category for state-sponsored workshops will help 

t respondents who were looking for such a category. Perhaps it 

should be included with the School-district sponsored 
category as “SCHOOL DISTRICT or STATE sponsored 
workshops or in-service programs. However, this solution 
will not help respondents who have difficulty distinguishing 

* between the first two answer categories. It could even 

aggravate that problem. 






Ilf. p SCHOOL sponsored workshops or in-service programs 



One respondent had attended a workshop at Catholic University, which she 
reported under “SCHOOL sponsored workshop.” She wasn’t sure about this 
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though. She looked at the next category (university extension or adult education 
courses) and wondered if it belonged there. She decided to keep it in the school 
sponsored workshop category, but she wondered about this, even as she was about 
to go on to question 3 1 . 






Teachers who had received their teaching certificates through the alternative 
certification program had difficulty determining if they should mark this category 
or not. On the one hand, they were “university extension” courses. On the other 
hand, they were also in a program. As often happens in cases like this, one 
respondent who had recently completed courses towards his alternative certificate 
marked this category and another didn’t. 

Recommendation: Reword this category to “University extension and adult 

education courses, including alternative certification courses,” 
if it is supposed to include these courses or to “University 
extension and adult education courses, excluding alternative 
certification courses,” if it is not supposed to include these 
courses. 






Some respondents didn’t know what “professional growth activities sponsored by 
professional associations” were. There were problems with understanding what a 
“professional growth activity” was. For instance, one respondent when she came 
across this category, exclaimed, “I don’t have any idea what they are talking about 
when they say “professional growth activity. What’s a growth activity?”. The 
respondent couldn’t even figure out what this was by a process of elimination 
because she said that so many other things were listed here that she hadn’t the 
faintest idea what a growth activity was. 

There were also problems understanding what “professional association” meant. 
One respondent said that a professional association to him would be the doctor’s 
association or a lawyer’s association or an association of referees. In other words, 
he said, an association of people OTHER than educators. 

He wasn’t alone in wondering what constituted a professional association, for 
another respondent questioned if she should count her teacher’s convention here. 



She decided to, demonstrating once again that when respondents are confused over 
something, they often draw opposite conclusions. Also, it is interesting to note 
that she also reported having attended the teacher’s convention earlier under the 
first category on the Private Teacher Questionnaire, “Workshops or in-service 
programs sponsored by an organization with which this school is affiliated.” If the 
distinction between these two rests with the distinction between “workshops or in- 
service programs” in the one category and “professional growth activities” in the 
other, this distinction was missed. 

One respondent defined this category as “activities that the Department of 
Education would present.” 

Recommendation: Provide concrete examples from which respondents can draw 

an understanding of what this category is supposed to include, 
for example, “Professional activities sponsored by 
professional associations, such as conferences, (and a list of 
more examples), etc.” Also, delete the word “growth,” since 
respondents do not understand it. 



. □ Other curriculum committee 
o Committee on selecting textbooks orimateiiaJs • • 



One respondent reported the same art curriculum under both the “other curriculum 
committee” and the “committee on selecting textbooks or materials.” It is not 
clear if this was appropriate or not. What is the distinction between these two 
categories? 

A private school respondent questioned the “committee on selecting textbooks” 
category. Since theirs was a small school, the respondent had just met with the 
principal to select a new science curriculum. She decided that this wasn’t 
technically a committee, so she left this answer category blank. It is not clear if 
she should have or not. 

Recommendation: Further define these categories. 



Additional categories 



When respondents were done answering this question, we asked them if there were 
any categories they thought should be included here that weren’t. One respondent 
suggested that “activities in the school with children” be included. She pointed out 
that volunteering to conduct extra activities in the school with the children is 
important. 



1 31* ; Since the end of last’ school year, have yon participated in any in-service or 

liliiiilijpr^ 



Respondents had trouble distinguishing between the categories in this item. For 
example, one respondent said, “I’d say b, methods of teaching in question b, 
was kind of, of, a, a heading for d [Student assessment] and e [Cooperative 
learning] ‘cause I, I included both of those, um, under that.” 

Several respondents had problems distinguishing (b) Methods of teaching your 
subject field from (a) Uses of educational technology for instruction. One 
respondent questioned whether she should report the computer in-service she had 
reported in (a) in (b) as well. After reading the question several times, she 
decided that category (a) was for her “benefit only,” whereas (b) benefitted 
students directly. She answered this question correctly, but for the wrong 
reason. Given that she was a special education teacher, I’m assuming she 
shouldn’t have even considered reporting the computer program in (b), since it 
wasn’t her subject field. 

When it came to (b) Methods of teaching your subject field, a couple of 
respondents included meetings they had attended in their answer here. It wasn’t 
clear if meetings were supposed to be included here or not. 

There was also trouble understanding the definition for (c) In-depth study in your 
subject field. One respondent interpreted this category as meaning an in-depth 
analysis or critique of one’s subject field or in his words “deep kind of picking 
apart at, uh, you know, how well it’s working.” He said that there had been a 
“general study, just kind of touching of the water,” but since the category 
specified “in-depth,” he didn’t think the general study qualified. Another 
respondent had trouble because her subject field was elementary education, and 



at first, she included the same workshops that she had reported in both (a) and 
(b) in (c). She said, “The more I learn about how to teach, the better I am at 
my field because teaching is my field.” However as she thought more about 
this, she came to understand that (c) was actually referring to content. She said, 
“...I guess maybe this is meaning content, as, have I learned about math? Like, 
how to do math, more than how to teach math.” However, she didn’t change 
her answer. She reported the same “more than 32 hours” in (c) as in (b). She 
said, “Even if they want to say these 32 hours can’t be inclusive with these 32 
hours, I would have 64 hours to cover both of these.” 

One respondent demonstrated that scales are seen as meaningful by respondents. 
As a follow up to whether or not they have participated in any of the listed 
programs, respondents are asked how many hours the program lasted and they 
are provided with three answer categories in which to respond: “8 hours or 
less.,” “9-32 hours,” and “More than 32 hours.” After looking these categories 
over, the respondent said, “Obviously 32 is the best answer, and if you don’t 
have 32 hours, you’re not competent in that.” 

Generally, respondents didn’t count the actual number of hours they spent 
participating in in-service or professional development programs; instead they 
tended to use words like ‘say’ and ‘probably’ to indicate that they weren’t 
calculating an exact number, but estimating instead. They said things like, 

“...I’ll say ‘8 or less’ because it’s informal; ...so that would be 8 hours or less. 
Probably 2 hours; I’m going to say 9-32 hours; yes, probably 9 to 32.” 
However, this answering strategy was probably adequate given that the relatively 
large ranges provided. 

A couple of respondents were confused by the wording of the follow-up 
question. The reference to ‘the program’ in the follow up question confused 
them. One respondent said, “I read that question wrong because it says ‘How 
many hours did the program last?’.” They are talking about one particular in- 
service. I am trying to think here, one in-service, or are they talking about the 
entire program on this particular topic ...because we have had a number of in- 
service programs.” Another respondent said, “I kind of took this at first as just 
focusing on in-service day... or just days... and not the program, and the program 
you know is a whole year kind of ...” . 

Recommendation: Clarify the distinction between the categories and whether 

these categories are supposed to be mutually exclusive or 
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not. Also, remove the confusing reference to ‘the 
programs’ in the follow-up question: How many hours did 
you receive? 

v . NQTE\i&Bsvi^^ 

• framing topics listed in item 31 on page 18. 

|3p : :li|K<^e5^ye . jhe|§$ 

f programs* described in item 31, in which you have participated since last . 

| school year>.: •’■•••• : .••••"?••• • | 

A few respondents expressed difficulty answering this question. Having 
participated in a number of different in-service or development programs led to 
their wanting to answer the categories differently depending on the particular in- 
service or program. Instead they needed to “average” the responses, which they 
found difficult to do. 



b . bn teaching 

c: Caused me to change my teaching practices jj 



These categories confused many respondents. For instance, one teacher used to 
think that children should sit in their seats and be quiet, now she lets them 
cooperatively learn and be talkative. She wondered if that would count as 
having changed her views on teaching or as having changed her teaching 
practices? 

In addition, a couple of respondents had interesting interpretations of the 
category “changed my views on teaching.” One respondent thought that if she 
said that the experience had caused her to change her views on teaching, that she 
was in effect saying that her previous view had been bad or wrong. Naturally, 
she was reluctant to admit to this. Another voiced the opinion that his view of 
teaching was always in effect changing, which to him meant that the programs, 
therefore, weren’t ultimately responsible. 

Recommendation: The distinction between (b) and (c) needs to be made clear. 

But more importantly, some real thought needs to go into 
determining what value these data provide when respondents 
“average” their responses over more than one program. It 
appears that the only way to get really meaningful data 
would be to ask this question with regard to each and every 



program respondents have taken, which would obviously be 
an overwhelming request. 



yoii: recfeij^^ 

l$^^?ln-service:ediicatlon or professional development in your MAIN teaching f® 
|l|;|f assignment 

For the most part, respondents did not speak in terms of the current school year. 
One respondent even went so far as to answer this question in terms of the future, 
e.g.., “If there were classes that would require a fee on the workshop days, I think 
that would be covered.” Also, one respondent was confused by the use of the 
word “support” in this question. She did not understand what kind of support the 
question was referring to until she read through the answer categories. 

Recommendation: Reword the question so that it highlights the reference period 

“During the current school year, what types of support have 
you received for in-service education or professional 
development in your MAIN teaching assignment field? 



o Released time from teaching 



One respondent was not sure what “released” time from teaching was until after 
she read the definition for “scheduled” time. Then she was able to derive a 
working definition for “released” time. 

Recommendation: Include a parathetical definition to help respondents 

distinguish “released” from “scheduled” time. 



o - Seheddied time 



One respondent decided that “scheduled time” included the teacher planning 
period. She explained that she considered developing a curriculum or a lesson 
during the teacher planning period as a form of professional development. It is not 
clear if this is true or not. 

Recommendation: Clarify this category. 



Q.y Tuition and/or fees ' 



One respondent had received a grant and wasn’t sure if that should be considered 
support for tuition and/or fees or not. Technically, she didn’t think it should be 
considered tuition or fees, which is why she immediately responded “no” after 
reading this answer category. Upon reflection, however, she decided that the grant 
did represent “support,” so she decided to mark this category. 






A number of respondents didn’t know what this category meant. Generally, they 
didn’t mark this category, since they didn’t know what it was. One respondent, 
however, interpreted the word “credit” to mean Maryland State Department 
Education credit. On the one hand she said that this meant, could she apply the 
course that she took for her recertification? On the other, it meant did her principal 
give her credit for professionally developing. She marked this category, saying 
that he did. 

Recommendation: Clarify this category. 



SECTION C-TEACHER1TOMNG ffgg " "... / . ;•••. | 

There was at least one respondent who objected to the organization of the teacher 
training section. She was an elementary teacher who taught mainly science and 
math. However, most of her science and math experience was gained from 
workshops and in-service educational experiences, not from formal classroom 
courses. Because she didn’t realize that eventually she was going to be asked 
about her professional development experiences, she became aggravated at having 
to report that she was a science teacher in question 29 after having reported she 
had not taken any science courses in question 28. After finishing question 29, she 
said, “That’s all they wanted to know, that I do teach it, see that upsets me because 
here [she is referring to question 28], I’m saying I have no background in math or 
science, and all they ask is do you teach it. Yes. And that makes me think that 
she’s not accredited. She’s not well versed in math and science knowledge, yet 
she teaches it. ‘Grunts’ to her. It ticks me off.” However, she calmed down once 
she was able to report her training in questions 30 and 3 1 . 

Recommendation: Because everyone’s most recent training experience, even that 

of new teachers, is the in-services and workshops they have 



just completed, perhaps the questionnaire should ask 
questions 30-32 before 15-29. At the very least, questions 30- 
32 should be asked before 28 and 29. 
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Many young teachers did not know the rules governing retirement eligibility. 
Retirement for them was so far in the future that they didn’t know much about it. 
For instance, one respondent said that she didn’t know if she was eligible to retire 
after 20 or 25 years of service, but she was sure she could after 30 years. Since she 
wasn’t sure one way or the other, she decided to hedge her bets by using the 25- 
year figure. Another respondent said that she could retire after 30 years of service. 
Two more respondents “guessed” it was 30. Another didn’t have any idea, so she 
didn’t answer this question at all. Another said that this was a silly question 
because she didn’t even want to think about retiring. She didn’t know how old you 
had to be in order to retire, but she assumed that she would be eligible to retire 
when she was around 55 (years old). A couple of respondents said that there 
wasn’t a mandatory age for retirement, like there had been in the past. They 
interpreted this question to be asking them when they could draw social security. 
One respondent even asked permission to write that date in. 

Recommendation: What are the rules governing retirement eligibility? Providing 

them to respondents would be helpful. However, an even 
better idea would be to not ask this question of new teachers. 
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Table 1. Number of teachers interviewed by school type and teaching experience. 



School Type 


■ New 


Experienced 


Alternative 

Certificate 


Total 


Public 


5 


5 


5 


15 


Private 


3 


2 


0 


5 


Total 


8 


7 


5 


20 



Table 2. Percent of respondents who misexecuted the skip instruction by form, question, 
and various question features. 



Form 


Question 


Skip 

Instruction 

Type 


Res- 

ponse 

Opt- 

ions 


Connec- 

tive 

Symbol 


Skip 

Instruc- 

tion 

Loca- 

tion 


Per- 

cent 

Mis- 

exe- 

cuted 

(N) 


Private 

Teacher 


1 . Main 
Assignment 


inter- 

mediate 


many 


bracketed 

arrow 


right 


30% 

(20) 


Private 

Teacher 


6. Main 
Assignment 
Year Before 


modified 

conven- 

tional 


many 


bracketed 

arrow 


right 


35% 

(20) 


Private 

Teacher 


9. Public 
Teacher 


salient 


yes/no 


numerous 

single 

arrows 


beneath 


0% 

(20) 


Private 

Teacher 


15d. Second 
Major 


salient 


yes/no 


single 

arrow 


beneath 


10% 

(20) 


Private 

Teacher 


11a. Breaks 
in Service 


inter- 

mediate 


none/# 


single 

arrow 


right 


25% 

(20) 


Decen- 

nial 

Short 

Form 


7.Household 

Elsewhere 


conven- 

tional 


yes/no 


single 

dash 


right 


25% 

(55) 
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Table 3. The question sequence of three respondents who exhibited real difficulty 



Respondent 13 


Respondent 1 9 


Respondent 2 1 


la 


la 


la 


5 


5 


lb 


2 


lb 


2 


5 Had already done 


2 


5 


4 


5 Had already done 


3a 


5 Had already done 


3a 


3c 


6 


4 


4 


9a 


3b 


5 Had already done 


10a 


3c 


6 


10b 


5 Had already done 


7 


7 Didn’t answer 


6 


8a-e 


because of note 


9a 


9a-c 




10a 


10a 


8a Didn’t answer 


7 Didn’t answer 


10b 


because of note 


because of note 


lla-d 


11a 




12 


lib 


8a-e Didn’t answer 


13 


12 


because of note 


14 


13 




15a-d 


14 


9a Had already done 


15f 


15a-d 


9b 




15f 


11a 






12 






13 






14 






15a-d 






15f 
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1. How do you classify your MAIN assignment at 
THIS school (i.e., the activity at which you 
spend MOST of your time) during this school 
year? 

Mark (X) only one box. 



0101 1 □ Regular full-time teacher — Skip to item 5 



2 D Regular part-time teacher 



»n 






Itinerant teacher (i.e., your 
assignment requires you to provide 
instruction at more than one school) 

Long-term substitute (i.e., 
your assignment requires that you fill 
the role of a regular teacher on a 
long-term basis, but you are 
still considered a substitute) 



Continue 
> with 
item 2 



sQ Short-term substitute 



6D Student teacher 

7D Other professional staff (e.g., counselor, 
librarian, curriculum coordinator, social 
worker) 



eD Administrator (e.g., principal, assistant 
principal, director, head) 

9 D Teacher’s aide 

io □ Support staff (e.g., secretary) 



IF YOU MARKED BOX 5, 6, 7, 8, 9, OR 10 ABOVE, PLEASE STOP NOW AND RETURN THIS QUESTIONNAIRE 
TO THE BUREAU OF THE CENSUS. THANK YOU FOR YOUR TIME. 



Figure 1. Example of conventional skip instruction from the 1991-92 Field Test 
Public School Teacher Questionnaire. 
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la. How do you classify your main assignment at THIS school (i.e., the 

activity at which you spend most of your time) during this school year? 

Mark (X) only one box. 



0020 



iD 

*□ 

*□ 

«□ 



Regular full-time teacher ► ^ O to hem 5, jp age 5 . 1 



Regular part-time teacher 



Itinerant teacher (i.e., your assignment requires you to 
provide instruction at more than one school) 

Long-term substitute (i.e., your assignment requires 
that you fill the role of a regular teacher on a long-term 
basis, but you are still considered a substitute) 



* I#/ 



to item 2, page 4. 



b. 



0025 



.□ 

n 

fsD 

.□ 

ioD 

oiD 



Please STOP now and return this questionnaire to 
Lthe Bureau of the Cen sus. Thank you for your ti me. 

Administrator (e.g., principal, assistant principal, director, school head) 

Library media specialist or librarian 

Other professional staff (e.g., counselor, curriculum coordinator, social worker) 
Support staff (e.g., secretary) 



Short-term substitute 
Student teacher 
Teacher aide 



V 

Do you TEACH any regularly scheduled classes at this school? 

If you work as a library media specialist or librarian at this school, do not 
include classes in which you teach students how to use the library (e.g., library 
skills or library research). 

id Yes ► GO to item 2, page 4. 

2 D No ► i- Please STOP now and return this questionnaire to 

.the Bureau of the Census. Thank you for your time. 



6. What was your MAIN activity the year before you began teaching at the 
elementary or secondary level? 

Mark (X) only one box. 

0060 iD Student at a college or university 

2 D Caring for family members 
3 D Working as a substitute teacher 
«D Teaching in a preschool 
sD Teaching at a college or university . 



-► 1:60 to item 9a, page 7. 



■> ^Continue with item 7. 



6D Working in a position in the field of education, but not as a teacher ■ 

7 D Working in an occupation outside the field of education ► §GO to item 8a,page6 

sD Military service 



sD Unemployed and seeking work 
10 D Retired from another job 



-> |GO to item 9a, page 7. 



Figure 2. Examples of questions with a relatively complex structure from the 1993-94 
Private School Teacher Questionnaire. Question 1 uses an intermediate 
skip instruction. Question 6 uses a modified conventional skip instruction. 
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93. Have you ever worked as an elementary or secondary teacher in a PUBLIC SCHOOL? 



0090 



r 



iD Yes 
2 EH] No 



GO to item 10a. 



b. How many years did you teach FULL-TIME in public schools? 

Record whole years, not fractions or months. If less than 4 months, mark "None". 



0095 



iCD None or . 



Years 



C. How many years did you teach PART-TIME in public schools? 

Record whole years, not fractions or months. If less than 4 months, mark "None." 



0100 



jO None or . 



Years 



d. Did you have a second major field of study? 



0185 




D Yes 
CD No 



jGO to item ISf. 

▼ 

e. What was your second major field of study? 

Record the two-digit field code from the list on page 10 and the field name. 



0190 



Code 



Second major field 



Figure 3. Examples of questions with a relatively simple structure from the 1993-94 
Private School Teacher Questionnaire. Both questions use the salient skip 
instruction. 
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» 



» 



» 



» 



11a. Since you began teaching, how many times have you had a 
break in service of one year or more? 

on5 oCU None ► f GOto item 

OR 




Breaks in service 



b. Were any of these breaks due to a reduction-in-force or a lay-off? 



0125 id Yes - How many? ► 0130 

2 d No 



» 



» 



Figure 4. An example of a question containing a combination of complex and simple 
9 question features. 



» 




Does this person have another 
r residence? 

i 0 Yes 0 No-Go to next person. 

gg ;> How much time does this person spend 
r at the other residence? 

0 Half of the time or less than half 
i of the time 

0 More than half of the time 
If more than half the time, enter 
the address below. 

House/Building Number 



Street Name 

I M ! : i U i 

Apartment Number 

Mm 

City 

M M , ; i i 

State 

| i 1 i I i i I • 

Zip Code Include + 4 if known. 



Figure 5. 



An example of a question from the decennial short form that employed a 
skip instruction. 
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